Capacity Bound-free Web Warehouse

Web cache technologies have been developed as an extension of CPU cache, by modifying LRU (Least Recently Used) algorithms. Actually in web cache systems, we can use disks and tertiary storages since access time of disks (or even online tapes) is still shorter than time required for retrieving web pages from origin severs. Thus, we can remove the restriction of cache size that has been the most severe condition for designing cache algorithms. We still need to determine the priority of data for efficient processing. In this paper, the concept of Capacity Bound-free Web Warehouse (CBFWW) will be introduced. There are the following assumptions in conventional web cache systems. (1) Priority of each object corresponding to web contents is simply determined by using queue. (2) Each object is independent. (3) Transparency of cache is assumed, where a user cannot know the contents. As the communication speed of the web is very slow, we can use complicated algorithms to determine the priority. Priority of newly retrieved documents is determined by computing their similarities to the documents with known priorities. Although objects with high priority are usually stored in fast access storage, we will discuss how to handle large objects. In conventional database systems, usage information like priority is hidden to the users. As in web-based applications, only a small fraction of data becomes hot spot and hot spots are changing very rapidly, self-organizing property using dynamically changing priority is important. As web data are very much complicated, we have to handle mutually related data objects. Links and structures of documents are Permission to copy without fee all or part of this material is granted provided that the copies are not made or distributed for direct commercial advantage, the VLDB copyright notice and the title of the publication and its date appear, and notice is given that copying is by permission of the Very Large Data Base Endowment. To copy otherwise, or to republish, requires a fee and/or special permission from the Endowment. Proceedings of the 2003 CIDR Conference also factors to determine the priority. As a large amount of data is stored, we should use the contents like database systems, not like conventional cache systems. Therefore transparency is not assumed. We need mining functions to analyze the usage patterns. We can retrieve objects with usage data which were not possible by cache or database system. The whole systems should have functions of cache, databases and data warehouses. We can further add useful functions.